System for Performing On-Line Transaction Processing and On-Line Analytical Processing on Runtime Data

ABSTRACT

An in-memory computing system for conducting on-line transaction processing and on-line analytical processing includes system tables in main memory to store runtime information. A statistics service can access the runtime information using script procedures stored in the main memory to collect monitoring data, generate historical data, and other system performance metrics while maintaining the runtime data and generated data in the main memory.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/243,567 entitled “Method and Apparatus for Monitoring an In-memoryComputer System,” filed on Apr. 2, 2014, now U.S. Pat. No. ______, whichclaims priority from U.S. Provisional Patent Application No. 61/908,616,filed on Nov. 25, 2013, and which is related to U.S. patent applicationSer. No. 13/088,921, filed on Apr. 18, 2011, now U.S. Pat. No.8,600,955, both of which are incorporated by reference herein.

BACKGROUND

The present disclosure relates generally to computer systems and inparticular to in-system monitoring of multiprocessor computer systems.

Unless otherwise indicated herein, the approaches described in thissection are not prior art to the claims in this application and are notadmitted to be prior art by inclusion in this section.

A typical business enterprise comprises a large number of organizations(marketing, engineering, production, supply, sales, customer service,and so on). Large volumes of data are typically generated and collectedby these many organizations.

Business intelligence (BI) and business warehousing (BW) toolsconventionally are built on a database architecture where the data iscollected and stored onto disk storage systems and subsequently readfrom disks (e.g., hard disk drive units) comprising the disk storagesystem for analysis. Conventional architectures also separate thefunction of transaction processing and analytical processing.

On-line transaction processing (OLTP) is typically characterized by alarge number of short on-line transactions (INSERT, UPDATE, DELETE).OLTP systems in an enterprise are the source of data for the rest of theenterprise. Various organizations in an enterprise typically connect toan OLTP to record their activities. For example, a manufacturing groupmay connect to an OLTP system to input data into a manufacturingdatabase such as incoming parts, production units, tracking of defects,and so on. A sales department may connect to an OLTP system to inputdata to a sales database.

On-line analytical processing (OLAP), by comparison, constitute a userof the data collected and stored in OLTP systems. Whereas OLTP may beviewed as a collector of raw data, OLAP may be viewed as a user of theraw data. OLAP queries are often complex and involve aggregations of thedata stored in one or more OLTP databases. An OLAP database typicallystores aggregated, historical data. OLAP is typically characterized by alower volume of transactions as compared to OLTP.

There is always huge demand for real-time reporting that can leveragereal-time data and provide improved decision making capability byreporting from transactional and operational systems. The success of abusiness may depend on how quick a reliable and smart decision can bemade based on information available at that moment. Real-time computingsystems have been evolving to meet these needs. One such system is basedon an architecture known as in-memory computing.

In-memory computing can parse and analyze data in a matter of minutes toseconds as compared to conventional computing architectures which mayrequire days to weeks. In-computing architectures are highly integratedsystems. Maintaining and otherwise supporting such systems requireequally fast response times to detect and assess changes in the systemthat may degrade performance.

These and other issues are addressed by embodiments of the disclosure,individually and collectively.

SUMMARY

A method and apparatus for monitoring an in-memory computer systemincludes performing on-line transaction processing and on-lineanalytical processing in the in-memory computer system. Results of theprocessing may be stored in main memory of the in-memory computersystem. Runtime information relating to the on-line transactionprocessing and on-line analytical processing may be stored in systemtables in the main memory. Information from the system tables can becopied as monitoring data by executing one or more script proceduresstored in the main memory.

In embodiments, executing script procedures in the main memory canperform data analysis on the monitoring data in the system tables togenerate a plurality of system performance metrics.

In at least certain embodiments, the script procedures can be executedin the main memory to evaluate the monitoring data according to alertcondition definitions and to write one or more alerts to alert tablesstored in the main memory when an alert condition is detected.

The following detailed description and accompanying drawings provide abetter understanding of the nature and advantages of the presentdisclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an in-memory computing system in accordancewith the present disclosure.

FIG. 2 represents a high level system diagram of an in-memory computingsystem in accordance with the present disclosure.

FIG. 3 illustrates an example of an alert in accordance with anembodiment.

FIG. 4 illustrates an example of an alert that can be communicated.

FIG. 5 shows an example of a data table definition in accordance with anembodiment.

FIG. 5A shows an instantiation of the data table defined in FIG. 5.

FIG. 6 is an example of an array definition in accordance with anembodiment.

FIG. 7 is an example of an array definition for alerts.

FIG. 8 represents a high-level diagram of a statistics service inaccordance with the present disclosure.

FIG. 9 is a block diagram of an example in-memory computing system inaccordance with the present disclosure.

FIG. 10 is a block diagram of an example implementation of an in-memorycomputing system with statistics service system monitoring in accordancewith the present disclosure.

FIG. 11 is a flowchart of a method for monitoring an in-memory computersystem in accordance with the present disclosure.

DETAILED DESCRIPTION

Described herein are embodiments for in-system monitoring of in-memorycomputing architectures. In the following descriptions, for purposes ofexplanation, numerous examples and specific details are set forth inorder to provide a thorough understanding of the present disclosure. Itwill be evident, however, to one skilled in the art that the presentdisclosure as defined by the claims may include some or all of thefeatures in these examples alone or in combination with other featuresdescribed below, and may further include modifications and equivalentsof the features and concepts described herein.

With reference to FIG. 1, in embodiments of the present disclosure anin-memory computing system 100 for an organization can be interfacedwith various business tools of that organization. One class of businesstools is business applications 134 used by the various operating groupsin the organization. Business applications 134 can communicate with thein-memory computing system 100 to serve as a repository for the datagenerated by those operating groups. Typical business applications thatthe operating groups execute include applications such as PLM (productlifecycle management), CRM (customer relationship management), PPS(product production system), ERP (enterprise resource planning), and soon. These applications typically generate data (referred to as“transaction data”) that can be stored in the in-memory computingsystem. For example, transaction data may be information relating to thesale of widgets such as where the sale occurred, when the sale tookplace, the sale price, and so on. Transaction data may be informationabout the production of widgets such as how many widgets were producedat a given manufacturing site, information about the raw materials usedto make the widgets and so on.

Another class of business tools is business intelligence (BI) analysisapplications 132 (also referred to a BI tools). This class of businesstools typically provides historical, current, and predictive views ofbusiness operations. BI analysis applications 132 play an important rolein the strategic planning process of the organization. Theseapplications treat the collected and analyzed data as businessintelligence in the areas of customer profiling, customer support,market research, market segmentation, product profitability, statisticalanalysis, and inventory and distribution analysis, production andmanufacturing tracking, and so on in order to aid in the decision makingprocess. Common functions include reporting, online analyticalprocessing, analytics, data mining, business performance management,benchmarking, text mining and predictive analytics, and so on. BIanalysis applications 132 can communicate with the in-memory computingsystem 100 to access the data provided by the business applications 134and data generated from analyses conducted on the “raw” data provided bythe business applications.

Referring to FIG. 2 for a moment, a system level block diagramillustrates a typical arrangement of embodiments of the presentdisclosure. Applications 215 and 231-235 represent instantiations of thebusiness tool 132, 134. The applications 215 and 231-235 may communicatewith the in-memory computing system 100 in any of a number of ways. Acommon configuration is IP (internet protocol) based communication. Forexample, applications 215 may operate locally within a local areanetwork 220. The in-memory computing system 100 may be locally connectedon the local area network 220. Other applications 231-235 may be locatedoutside of the local network can communicate with the in-memorycomputing system 100 over a suitable communication system such as theInternet 230.

In embodiments, the in-memory computing system 100 may include a dataprocessor subsystem 201 comprising a plurality of data processing units.A main memory subsystem 202 may comprise various forms of read/writerandom access memory, including volatile memory such as DRAM andnon-volatile memory such as FLASH memory, and so on. The main memorysubsystem 202 may also include read-only type memory such as ROM memory.A distinction between main memory and disk-based memory is the formerdoes not require disk seek operations (e.g., rotating a disk, moving aread/write head into position, etc.) and thus can access datasignificantly more quickly.

The in-memory computing system 100 is a computing architecture in whichdata is moved off disk storage and into main memory, thus avoiding theneed (and delay) to run disk-seek operations each time a data look-up isperformed. As can be appreciated, this approach significantly increasesperformance. For example, the tables comprising a database can be storedin main memory (e.g., RAM, flash memory, and the like) as opposed todisk storage such as hard disk drives, thus increasing data look-upperformance.

To further improve data look-up speed, data tables in the in-computingsystem 100 can be stored in column-oriented fashion, rather thanrow-oriented. For example, a customer record might have name, storelocation, purchased item, and price as relevant fields (four fields). Atable of such customer records might have the following data:

name store item price John S1 shoes 100 Joe S4 book 20 Mary S1 pens 10Sam S2 paper 50 Dale S2 shirts 45In a row-oriented storage scheme, the data comprising the table would bestored in continuous locations in main memory or on disk in thefollowing sequence:

John S1 shoes 100 Joe S4 book 20 Mary S1 pens 10 Sam S2 paper 50 Dale S2shirts 45

In a column-oriented storage scheme, the data would be stored in thefollowing sequence:

John Joe Mary Sam Dale S1 S4 S1 S2 S2 shoes book pens paper shirts 10020 10 50 45

Where analytical processing that requires the computation of aggregatecalculations constitutes a major component of the workload in thein-computing system 100, column-oriented storage would be advantageousbecause it is more efficient to perform aggregate operations on data(e.g. summing of data) when the data is stored in sequential memorylocations.

The in-memory computing system 100 may nonetheless include a storagesubsystem 203 comprising one or more mass storage devices such as harddisk drives and the like, to store the operating system, administrativeinformation (e.g., login accounts), and the like. A network interfacesubsystem 204 can provide communication between the in-memory computersystem 100 and applications 215 and 231-235 over various electroniccommunication media and protocols. In one embodiment, the networkinterface subsystem 204 can include functionality to communicate over atelecommunication network. A system of buses 205 can interconnect theforegoing subsystems, providing control lines, data lines, and/orvoltage supply lines to/from the various subsystems. The in-memorycomputer system 100 may include a suitable display(s) 212 and inputdevices 211 such as a keyboard and a mouse input device.

Returning to FIG. 1, in embodiments, the in-memory computer system 100includes various processes 102 (e.g., executed by the data processingunits 201) in communication with the business tools 132, 134. Forexample, the business applications 134 may provide data collected duringthe daily operations of the organization that would be stored in thein-memory computing system 100. For example, a CRM business applicationmay generate customer support data. A PPS business application maygenerate information about suppliers of parts, and so on.

Accordingly, one or more on-line transaction processing (OLTP) serversexecuting on the in-memory computing system 100 can communicate with thebusiness applications 134 to receive data from those applications; e.g.,customer purchases, manufacturing logs, telemarketing data, and so on.As explained above, the in-memory computing system 100 stores receiveddata in main memory 104 rather than on disk storage devices in order torealize improvements in data access speed by avoiding data seekoperations required of disk storage devices. Accordingly, the OLTPservers may store data received from business applications 134 intotables 104 a that are memory resident (as opposed to disk resident).

One or more on-line analytical process (OLAP) servers can access thedata stored in the data tables 104 a to analyze the data in the datatables to provide insight into the organization's business and tofacilitate planning of various operations such as manufacturing,marketing, customer support and so on. The OLAP servers may receive datadirectly from the business applications 134 as well as access the datatables 104 a to perform various analyses. Results produced by the one ormore OLAP process servers may include reports, new data tables for datamining, and so on. Such results can be stored back into main memory 104for subsequent access and further analysis. Storing the results backinto main memory 104 allows for quick access to the results without thedelays of disk storage devices.

Additional servers may be provided in the in-memory computing system100, for example, to support the operations of the OLTP servers and theOLAP servers. In some embodiments, an additional server can be used toschedule or trigger operations performed on data in the main memory 104.

Though the collected and analyzed data may be stored in main memory 104for performance reasons, the in-memory computing system 100 maynonetheless include a suitable backup or remote storage system in orderto provide permanent storage of the data. For example, a backup process106 may execute in the in-memory computing system 100 to read out datastored in main memory 104 to be stored in a backup storage system 106 a.The backup storage system 106 a may be co-located with the in-memorycomputing system 100, or may be remotely located (e.g., in a differentgeographic location).

The business intelligence analysis applications 132 can communicate withthe OLTP servers and the OLAP servers to access the “raw” data producedby the business applications 134 and collected by the OLTP servers, andto access analyzed results produced by the OLAP servers in order toprovide planners in the organization with information for makingdecisions. For example, business intelligence analysis applications 132can include a wide range of tool such as financial analysis, statisticalprocess control, and so on. Business intelligence analysis applications132 may be used to spot trends, identify patterns, interpret outliers,uncover unexpected relationships within the raw data collected by theOLTP servers, and the like. Business intelligence analysis applications132 may collect results from or otherwise interact with OLAP servers tocreate reports such as sales forecasts, manufacturing requirements, andso on.

In embodiments, the in-memory computing system 100 may include astatistics server process 112 executed by the data processor subsystem201 (FIG. 2). The statistics server 112 can communicate with the otherservers 102, for example, via a suitable inter-process communicationprotocol. The statistics server 112 can collect runtime information asmonitoring data from processes 102 executing in the in-memory computingsystem 100. In embodiments, runtime information can be collected fromthe operating system (OS) as well.

Runtime information may include state information about the state of aprocess 102. Run time information may include state information aboutthe in-memory computing system 100 itself. For example, stateinformation may include system information relating to the components ofthe in-memory computing system 100 such as memory usage information,processor load data, information about other processes that areexecuting, users who might be logged on, information about theperformance of the data processing unit of the data processing subsystem201, memory access speed data, IP packet traffic, and so on. Suchinformation may be collected by a system monitoring process havingsufficient access (e.g., supervisory access) to the operating system(OS) tables and other OS level data.

State information may include process-specific information for eachprocess 102. State information may include a process start time (if theprocess is a scheduled process), total run time, number of memoryaccesses, memory allocations made by the process, and so on. Runtimeinformation can be generated by a process 102 to indicate the occurrenceof events that might occur during execution of the process. For example,runtime information may include exception codes that a process 102 mightgenerate as a result of encountering an error in the data (e.g.,unexpected data format, illegal data value, etc.). An exception code maybe generated by a process 102 if an error is detected with a data tablethat the process is accessing. An exception code may be generated if aprocess 102 did not have sufficient time to complete a task, and so on.Process-specific information may include data about how much data iscollected by the process 102, how much data is generated by the process,etc. Such information can be collected by sub-routines within theprocess 102, or by process threads created (e.g., spawned, forked) bythe process.

In embodiments, the runtime information can be stored in system tables104 b in the main memory 104 corresponding to each process 102. Somesystem tables 104 b may be shared by two or more processes. Inembodiments, the statistics server 112 can collect the runtimeinformation by interrogating each process 102. The process 102 can thenaccess the appropriate system table(s) 104 b and provide the statisticsserver 112 with suitable responses. In embodiments, the statisticsserver 112 can collect the information from the system tables 104 b. Inan embodiment, the system tables 104 b can be in the form of databasetables. For example, the system tables 104 b can be relational databasetables accessed using an SQL query language.

The statistics server 112 can accumulate the collected data as historicdata in a statistics tenant 104 c in main memory 104 for a posteriorianalysis. The statistics server 112 can perform analyses on thecollected data to generate system performance metrics. For example, thestatistics server 112 can produce a history of memory usage. Examples ofperformance metrics can include a history of processing times of theprocesses 102, responses to user requests, and so on.

A suitable user interface can be provided to allow a user 136 to querythe historic data contained in the statistics tenant 104 c. Inembodiments, the user 136 can be notified with alerts to indicate theoccurrence of events. For example, FIG. 3 illustrates a portion of adisplay of an alert that can be presented on a suitable display device.

In embodiments, the statistics server 112 can notify the user 136 ofcertain alerts 118, for example, if immediate action is required. Anysuitable notification can serve as an alert 118. For example, referringto FIG. 4, the statistics server 112 can generate an email message thatincludes relevant information about the situation. Alert triggers can bespecified by the user 136 to control when an alert 118 is communicatedto the user. For example, available memory falling below 20% can be acriterion for sending an alert 118.

In embodiments, an external application can interface with thestatistics server 112 to gather the historic data that has beencollected by the statistics servers to conduct a more detailed analysis.For example, business intelligence analysis applications 132 aretypically designed to analyze business data collected by theorganization's various operations and make or facilitate strategicbusiness decision-making activities. Accordingly, the same businessintelligence analysis applications 132 may be adapted to identifyunderlying trends in the historic data and/or perform additionalanalyses on the historical and performance data to steer technicaldecisions regarding the performance of the in-computing system 100.

Consider a simple example: Suppose an organization has offices in Berlinand in California. Suppose a user in the Berlin office has scheduled abackup to take place each day at 00:30 AM. The statistics server 112 maycollect information about the occurrence of backup processing happeningeach day at 00:30 AM (Berlin time). Suppose the statistics server 112also collects data indicating slow response times of a process takingplace in the California office at around 10 AM. These two apparentlyindependent observations can be analyzed by a business intelligenceanalysis application 132. The business intelligence application 132 mayconsider the time difference between Berlin and California—Berlin isahead of California by 9 or 10 hours, depending on the time of year, andidentify a correlation between backups taking place in Berlin and whenthe process in California is being performed. The business intelligenceapplication 132 may then conclude that the observed slow response timesin California are due to the backup process taking place at around thesame time in Berlin. This result can be used by a system administratorto reschedule the backup activity in Berlin or the activity inCalifornia, or the administrator may consider upgrading the hardware(e.g., additional processors, memory, and so on).

The statistics server 112 may include a configuration file 114 toconfigure the activities of the statistics server. Configurationinformation can be input by the user 136 and stored in the configurationfile 114. In embodiments, the configuration information can specify manyaspects of the collection of performance data; the collection is highlyconfigurable. The configuration information can specify what data tomonitor and collect. The configuration information can specify acollection schedule for the statistics server 112. Since collectionactivities of the statistics server 112 draw on system resources of thein-memory computing system 100 (e.g., processor bandwidth, memorybandwidth), it may be desirable to schedule when the statistics serveroperates, for how long the statistics server operates, and how frequentthe collections are performed. The configuration information can specifythe format (e.g., data table schema) of output data generated by thestatistics server 112. Being able to specify the output format canfacilitate interfacing the output data with analytical tools (e.g.,business intelligence analysis applications 132). The configurationinformation can include triggering information to specify the conditionsfor when an alert 118 is communicated to a user. More generally, theconfiguration information can specify performing actions in addition toor instead of sending alerts 118. The configuration information canspecify rules for assessing the historic data collected by thestatistics server 112. The configuration information can specify/defineconditions and events, and the actions that should be taken when thoseconditions are met or when those events occur.

FIG. 5 shows an illustrative example of a data table definition for adata table that can be stored in the statistics tenant 104 c inaccordance with an embodiment and filled in by the statistics server112. In the particular example shown, the data table definitionspecifies a table to store utilization statistics for a host CPU. Thedata table definition specifies various data fields and their datatypes. FIG. 5A shows an example of an instantiation of a data table 500defined by the data table definition shown in FIG. 5. Columns 501-505are illustrated. Next will be a discussion of how the columns of thedata table can be populated.

Each of the columns 501-505 in the data table 500 is associated with anarray definition. FIG. 6 shows an example of an array definition 602that specifies how column 505 (target column) is populated. The arraydefinition 602 includes an “indexcolumn” key which specifies whichcolumn in the data table 500 to index on. In the example, the index ison the HOST column 503. The “sourceschema”, “sourcetable”, and“sourcecolumn” keys in the array definition 602 identify which of thesystem tables 104 b will serves as the source of data to populate thetarget column 505 in. In the example shown in FIG. 6, data from thesystem table SYS.M_HOST_RESOURCE_UTILIZATION is accessed, and inparticular the data is contained in the column calledTOTAL_CPU_SYSTEM_TIME. The array definition 602 further specifies thatthe target column 503 is updated every 60 seconds (see “intervals”). Inthis way, a history can be constructed.

Array definitions need not be used for populating columns in a targetdata table. For example, the array definition shown in FIG. 7 can beused to specify a trigger for an alarm. In embodiments, arraydefinitions can include arithmetic and logic formulas. The illustrativearray definition shown in FIG. 7 defines the criteria for an emailalert. There is a “label” key which specifies the message to be includedin the email. The array definition includes variables such asSHM_USED_SIZE and THRESHOLD_SHM_USED_SIZE_WARING_LEVEL_3 which arereplaced at runtime by corresponding actual values when the message isincorporated in the body of an email. In this way, the email text isdynamic and can be configured with the specific conditions of the systemthat the email is intended to convey. An “emailcondition” key specifiesan arithmetic expression as the criterion for sending the email alert.The value of the “name” key is sent in the email's subject field (it isjust plain text). The value of the “description” key can be used in asuitable graphical user interface (GUI) to explain the meaning of thealert to the user. In embodiments, formulas can incorporate other arraydefinitions; e.g., the “warning2” and “warning3^(”) keys are specifiedusing arithmetic expression that reference array definitions.

To further increase the speed and efficiency with system monitoring canbe performed, the various monitoring operations on the data stored inthe main memory 104 can be executed on the data without first removingit from the main memory 104. Accordingly, in some embodiments,operations on the data can be performed on the data in the main memory104 while it is still in the main memory 104. Performing such operationson the data in memory can reduce the computing resource overhead byavoiding at least some read/write operations of data in and out of mainmemory 104, as well as alleviating the need for some inter-processcommunication and logic. At least some, if not all, of the variousactivities described above as operations performed by the statisticsserver 112 and/or processes 102 can be accomplished using logic embeddedin query statements stored in and executed on data in the main memory104. In one embodiment, a statistics scheduler process in the in-memorycomputer 100 can be used to invoke one or more of the query statementsstored in the main memory 104.

In various embodiments of the present disclosure, simple and complexoperations and/or logic can be implemented as collections of querystatements that can include logical operators (e.g., OR, AND, NOT, ANY,etc.) stored in the main memory. The query statements can be structuredquery language (SQL) statements or commands. A set of SQL commands(e.g., an ordered or unordered list of several SQL commands) can besaved as an SQL script procedure. SQL script procedures can be saved inone or more systems tables in the database to which the SQL scriptprocedure belongs.

While many example embodiments in the present disclosure are describedin reference to what are commonly referred to as SQL commands and SQLscripts, one of ordinary skill in the art will recognize that otherquery and database languages can be used to generate commands that canbe compiled into script procedures to implement various data editing andlogic operations on data stored in tables in a database. Accordingly,the term script procedures can be used to refer to any collection of SQLand non-SQL commands stored in one or more tables in a database that canbe executed to operate on data (i.e., move, copy, edit, update, delete,analyze, evaluate, etc.) while it is still in the database. For example,one script procedure can be executed to select and copy data from onedatabase table in the main-memory component 104 to another databasetable without taking the data out of the main-memory component 104. Suchembodiments potentially achieve the benefit of increasing the speed andefficiency with which the various operations defined in the scriptprocedure can be performed on the data in the main memory 104.

As described herein, various monitoring operations begin with thestatistics server 112, or another process 102, retrieving runtimeinformation from the data stored in main memory 104. The statisticsserver 112 can retrieve runtime information from the system tables 104 bdirectly or by querying one or more processes 102, which in response,access the appropriate system table(s) 104 b to retrieve the runtimeinformation out of the main memory 104. The statistics server 112 canthen collect the runtime information for the various processes 102 bystoring it back into the main memory 104. In some embodiments, when thedata is stored back into the main memory 104 it is organized into tablesin the statistics tenant 104 c. However, reading data out of the mainmemory 104 just to write it back into the main memory 104 does requiresome finite amount of time and computing resources. Depending on thenumber and frequency of reads and writes out of and into the mainmemory, collecting runtime information can involve significant time andcomputing resources. Embodiments of the present disclosure can avoidsuch overhead by reducing or eliminating the need to read data out ofone or more tables in the main memory just to write it back into anothertable in the main memory. Instead, such embodiments can perform thecollection and analysis of runtime information, as well as the tableupdates/creation, without ever having to remove the runtime informationfrom the main memory 104. Specific example embodiments of statisticsservices, with implementations that include at least one scriptprocedure stored in the main memory 104, that can replace or enhance theperformance of the statistics server 118 are described in more detail inreferences to FIGS. 8 through 10.

FIG. 8 illustrates a high level block diagram of a statistics service800 that can perform data collection and analysis operations on datawhile it is still resident in the main memory 104 (FIG. 1). In someembodiments, the statistics service 800 can replace the separatestatistics server process 112 to further reduce the overhead associatedwith various system monitoring functionality described herein. In oneembodiment, statistics service 800 can be implemented using a statisticsscheduler process 810 and a number of scripts procedures, depicted hereas SQL scripts 820, stored in the main memory 104. Based on informationand settings in the configuration file 114, the statistics scheduler 810can cause one or more SQL scripts 820 to execute in the main memory 104.For example, one SQL script 820 can include one or more select commands(SEL) 825 to perform the necessary runtime data collection and analysison the data in the system tables 104 b on a periodic, or otherwisescheduled, basis. Another or the same SQL script 820 can include insertcommands (INS) 827 to store the collected and/or analyzed runtime datainto one or more tables in the statistics tenant 104 c and/or the alertstable 104 d. Accordingly, in such embodiments, none of the runtime dataneed ever come out of the database in the main memory 104 during thecollection, analytical, or storing processes.

In various embodiments, an SQL script 820 can include a set of SQLcommands saved in a database in the main memory 104. In one embodiment,the SQL script 820 can be saved in the database as metadata and exposedto other components of the in-memory computer system 100 as catalogviews. In another embodiment, the SQL script 820 can be stored in thedatabase in a set of system tables 104 b. The SQL script 820 can beaccessed using dedicated editors, commands, and syntax implementedspecifically to alter the scripts. For example, an SQL script 820 cancontain one or more SQL commands. An editing program in one or more ofthe business application 134, business intelligence analysis application132, or editing tools in an monitoring infrastructure can create, edit,view, run, and delete SQL commands in the script files. Execution ofindividual SQL commands can be independent of the execution of the SQLscripts 820 in the main memory 104.

To organize the historization of data, whenever the statistics scheduler810 initiates a particular SQL script 820, a time indicator can beassociated with the resulting data. For example, the data collected fromor alert data generated from the system tables 104 b can be associatedwith one or more timestamps (e.g., a start time and an end time) thatindicate a time of day and/or a date. The timestamps can then be used todetermine and organize a historical account of the runtime and/or alertdata. In embodiments, the timestamps can be included in a row of astatistics table 104 c along with the corresponding collected runtimedata or in a row of an alert table 104 d along with the generated alertdata.

In some embodiments, data resulting from the execution of an SQL script820 (i.e., collected runtime data or alert data) can be associated withan SQL script identifier that identifies the SQL script 820. Inaddition, the SQL script identifier and any resulting data associatedwith the execution of the SQL script 820 on data in the main memory 104can be stored with the associated timestamp. The time stamps can includean indication of the time at which a particular SQL statement or SQLscript started, and/or ended. Accordingly, the start time and/or the endtime can be used to determine various time characteristics. For example,the difference between the start time and the end time can interpretedas the amount of time it took for the particular SQL script 820 toexecute in that particular instance. Alternatively, the time stamp canbe used alone or in combination with other data to evaluate and/orgenerate a historical view of the collected or analyzed data.

As described herein, processes 102 can store runtime informationrelating to the on-line transaction processing and on-line analyticalprocessing stored in system tables 104 b. Such information can alsoinclude time indicators, such as timestamps, to provide a time framewith which to evaluate the historical performance of correspondingprocesses 102. One or more SQL scripts 820 can be executed in the mainmemory 104 to select specific runtime information from the system tables104 b. The selection of the runtime information from the system tables104 b can be based on a selection of a range of time indicators. Therange of time indicators can be hardcoded in the SQL script 820 or bebased on user preferences or system settings stored in the configurationfile 814. The same or different SQL scripts 820 can then be executed tostore the selected runtime information as monitoring data in statisticstables 104 c. Again, the same or different SQL scripts 820 can beexecuted on the data in the statistics tables 104 c to generate systemperformance metrics. The resulting system performance metrics can thenbe stored in one or more statistics tables 104 c, or other tables in themain memory 104, according to a corresponding SQL script 820. Over time,the SQL scripts 820 can generate and collect a history of monitoringdata and performance metrics in the statistics tables 104 c.

The collection of runtime data about the various processes 102 and/orthe system performance metrics can be used to collect a history ofsystem performance. The insertion of the collected runtime data and/orthe system performance metrics into one or more tables in the statisticstenant 104 c to develop a history of runtime data can include executingone or more SQL scripts 820 to select, analyze, and insert runtime datafrom one or more of the system tables 104 b into one or more measurementtables in the statistics tenant 104 c as monitoring data. In suchembodiments, runtime data is copied from a system table 104 b to a tablein the statistics tenant 104 c (e.g., a statistics table). In someembodiments, the runtime data can include analyzed data, such as thecalculated system performance metrics.

For example, the SQL scripts 820 can be used to populate column 505 inthe table illustrated in FIG. 5A according to the following SQLstatement:

INSERT INTO HOST_RESOURCE_UTILIZATION [TOTAL_CPU_SYSTEM_TIME] SELECTHOST_RESOURCE_UTILIZATION FROM M_HOST_RESOUCE_UTILIZATION;

In some embodiments, one or more of the SQL scripts 820 can includelogic for evaluating the runtime data or a system performance metric todetermine whether it should be stored in one or more alerts tables 104d. Such logic can be implemented as one or more SQL commands comprisingone or more logical operators (e.g., OR, AND, and NOT) to evaluate themonitoring data or metric with one or more predetermined alertconditions. The logic implemented in an SQL script 820 can compareconditions to determine whether a particular entry in a system table 104b, or a corresponding alert, should be inserted into an alerts table 104d. For example, the following SQL statement can be used to evaluate thecondition wherein the host resource utilization is greater than or equalto a particular threshold or NULL to determine whether to insert analert into the host resource utilization alert table:

INSERT INTO HOST_RESOURCE_UTILIZATION_ALERT [ALERT_EVENT] SELECTHOST_RESOURCE_UTILIZATION FROM M_HOST_RESOUCE_UTILIZATION WHEREHOST_RESOURCE_UTILIZATION >= HOST_RESOURCE_UTILIZATION_THRESHOLD ORHOST_RESOURCE_UTILIZATION = NULL

In embodiments, the SQL scripts 820 used for selecting and copyingruntime data from the system stables 104 b into one or more historytables in the statistics tenant 104 c can be separate from other SQLscripts 820 used for evaluating runtime data for entry into alertstables 104 d. Accordingly, SQL scripts 820 used for collecting runtimedata and SQL scripts 820 used for evaluating alert conditions in theruntime data can be edited independently of each other, thus givingusers flexibility to change the runtime data collection processeswithout changing or potentially causing errors in the alertdetermination processes. Similarly, changes to the alert determinationprocesses can be made without altering the runtime data collectionprocesses. Such features of the present disclosure potentially giveusers enhanced flexibility to implement custom data collection and alertprocessing without the need to change or recompile code for thestatistics server 112.

In some embodiments, once sufficient historical monitoring data and/orperformance metrics are collected, one or more SQL scripts 820 can beexecuted on the data in the statistic tables 104 c to evaluate variousalert conditions. For example, one particular SQL script 820 may beexecuted on the historical memory usage data stored in a statisticstable that corresponds to a particular process 102. Such an SQL script820 may evaluate the rate at which the memory usage of the particularprocess 102 changes with respect to time. If memory usage change rate isbelow or above a threshold rate defined in the SQL script 820 or inconfiguration file 814, an alert condition can be written to one or morealert tables 104 d. In one embodiment, sufficient historical monitoringdata can be a few as one record stored in the statistics tables 104 c.In another embodiment, sufficient historical monitoring data can includeone or more records stored in the statistics tables 104 c.

FIG. 9 is a schematic diagram of a specific example in-memory databasemanagement (IMDBM) system 900 that implements the statistics service801, in accordance with various embodiments of the present disclosure.The statistics service 801, can collect and evaluate information aboutstatus, performance, and resource consumption from all components orprocesses 102 in the IMDBM system 900. In one embodiment, the statisticsservice 801 can be implemented as SQL scripts 820 and a statisticsscheduler 810. The tables 904 and SQL scripts 820 can be resident in amaster index server (not shown). The statistics scheduler 810 can be athread running on a master name server (also not shown). In theparticular example shown in FIG. 9, the SQL scripts 820-1 and 820-2collect data and evaluate alert conditions. Accordingly, the SQL scripts820-1 and 820-2 are referred to herein as “data collectors” and “alertcheckers”, respectively.

In one embodiment, the data collectors 820-1 and the alert checkers820-2 can be invoked by the statistics scheduler 810 according topredetermined settings that can be stored in the configuration file 814.In addition, the data collectors 820-1 and the alert checkers 820-2 canbe invoked manually by a user through the monitoring infrastructuresystem 920.

The data collectors 820-1 can read runtime data from the system tables104 b, process the data, and store the processed data in the statisticstables 904 c, thus creating a runtime data measurement history. Asdiscussed herein, the tables 904, including the statistics tables 904 cand alert tables 904 d, can be resident in a main memory 104.

The alert checkers 820-2 can be scheduled in conjunction with orindependently from the data collectors 820-1, and can also read runtimedata from the system tables 104 b. While the alert checkers 820-2 areshown as reading runtime data directly from the system tables 104 b andnot from the statistics tables 904 c, in embodiments, the alert checkers820-2 can also read data from the statistics tables 904 c. Once thealert checkers 820-2 obtain the runtime data, the alert checkers 820-2can be further executed to evaluate one or more alert conditions 914-1.Evaluation of an alert condition can include collecting runtime data orcalculating a value from runtime data from the systems tables 104 b andcomparing it against one or more alert condition definitions in alertconditions 914-1.

Alert conditions 914-1 can include one or more alert conditiondefinitions that define one or more threshold values or status flagsthat indicate one or more conditions in the in-memory computer system100 or application 132 or 134. The alert conditions 914-1 can includeboth default and customized alert condition definitions. For example, acustom alert condition definition can be defined by a particular user(e.g., a system administrator) to monitor runtime data associated with aparticular problematic process (e.g., one of the OLAP servers 102 thathas been causing memory bloat in the data tables 104 a) to which arecent fix or software patch has been applied. In such an example, thecustom alert condition definition can include a threshold value for therate at which data can be added to the data tables. In contrast, defaultalert condition definitions can include specifications for thresholds orstatus flags that can be used to evaluate alert conditions in any of theprocesses 102 or in the in-memory computer system 100.

If an alert condition is detected, then a corresponding alert (e.g.,detail of the alert) can be written to the alert tables 904 d. Alertswritten to one or more alert tables 904 d can be accessed by monitoringtools in the monitoring infrastructure system 920, such as monitoring UI923, to generate an alert status message. In such embodiments, themonitoring infrastructure system 920 can access the data in thestatistics tables 904 c or alert tables 904 d using SQL statements.

In other embodiments, when an alert condition is detected or when analert is written to alert tables 904 d, the alert checkers 820-2, orsome other component of IMDBM system 900, can initiate a command to anexternal or integrated mail server 930 to send an alert message to oneor more users (e.g., system administrators) to inform them of the alertcondition according to settings in the email addresses 914-2 and alertconditions 914-1. The content and priority of the alert message (e.g.,email or short message service (SMS)), as well as the email addresses ortelephone number to which the alert message should be sent, can bedefined in the alert conditions 914-1 and email addresses 914-2.Accordingly, based on the alert condition detected, certain users, asidentified by their associated email addresses or telephone numbers, canbe included as recipients of the alert message.

For example, alert conditions that identify issues with the IMDBM system900 as a whole may be sent to one or more high level systemadministrators to increase the likelihood of an immediate resolution. Incontrast, for alert conditions that affect only isolated processes 102,an alert email can be sent to the individual technician who isresponsible for that particular process. Similarly, depending on theseverity, sensitivity, or security level of the alert conditiondetected, the alert checkers 820-2 can include or exclude the specificsabout the alert condition detected or determined by the correspondingalert checker 820-2. If the alert condition persists after an alertemail is initiated, then the alert conditions 914-1 and the emailaddresses 914-2 can specify that additional email messages be sent. Theadditional email may include escalating the priority of the email,including higher level or additional users as recipients of the alertemail, or including additional summary information about the alertcondition. In some embodiments, the frequency with which the alertemails are initiated can also be increased.

In one embodiment, the alert conditions 914-1, email addresses 914-2,the frequency with which the statistics scheduler 810 invokes datacollectors 820-1 and/or alert checkers 820-2, and other settings of theIMDBM system 900, can be edited by the monitoring configuration editor925 in response to user input received through the monitoringinfrastructure system 920.

Any interactions between the monitoring infrastructure system 920 andthe IMDBM system 900 can be conducted using one or more SQL statements.For example, when an administrator receives an alert email from theIMDBM system 900, the administrator can log into the monitoringinfrastructure system 920. Using one or more tools, such as themonitoring UI 923, the administrator can view the details of the alertcondition by manually accessing the alert tables 904 d or invoking thecorresponding alert checkers 820-2 to access the alert tables 904 d toevaluate the status of the alert condition. The administrator canevaluate the efficacy of various adjustments or fixes by invoking thealert checkers 820-2 to confirm that the alert condition either persistsor has been cleared.

FIG. 10 is a schematic diagram that illustrates how statistics services,such as 800 or 801 described in reference to FIGS. 8 and 9 respectively,can replace the statistics server 112 in an in-memory computing system100 for monitoring runtime data of the various processes 102 inaccordance with the present disclosure. As shown, the statistics server112 can be omitted, thus removing one sever process 102, which canimprove the performance of the in-memory computing system 100. Thestatistics service can be implemented as a statistics scheduler thread810, a configuration file 814, and a collection 104 e of scripts, alsoreferred to herein as script procedures, 820 resident in the main memory104.

As shown, the user 136 can define the configuration file 814. Asdescribed herein, the configuration file 814 can include listings ofspecific SQL scripts 820 that should be executed in the main memory 104to perform the corresponding data collection and alert conditionevaluation processes. The configuration file 814 can also includecorresponding frequencies with which each one of the SQL scripts 820should be executed. For example, the configuration file 814 may includethe specification of a particular SQL script 820 be executed to gatherruntime information regarding the OLTP servers and OLAP servers 102 fromthe statistics tenant 104 c every 60 seconds.

The implementation of the in-memory computing system 100 that includes astatistics service instead of a statistics server 112 allows for variousperformance improvements. One example improvement afforded by the use ofan embodiment of the statistics service in an in-memory computing system100 with a single index server, comprises the ability to copy data fromsystems tables 104 b to statistics tables 104 c or alert tables 104 dwith little or no inter-process communication. Most of the in-memorydata handling can be handled by one or more of the scripts 820 alreadyin the main memory 104 (e.g., data collectors 820-1 and alert checkers820-2). This decreases the time and computing resource required toperform data collection, historization, and alert condition checking,thus increasing the efficiency and stability of the in-memory computersystem 100.

FIG. 11 is a flow chart of a method 1100 for performing various systemmonitoring operations of an in-memory computer system, according toembodiments of the present disclosure. The method 1100 can begin at box1110 in which one or more processing units in an in-memory computersystem 100 can conduct various OLTP operations in one or more OLTPserver processes 102. As described herein, the OLTP server processes 102can store data, typically transaction data, in one or more data tables104 a in the main memory 104 of the in-memory computing system 100.

In box 1120, one or more of the processing units can perform variousOLAP operations in one or more OLAP server processes 102. The OLAPserver processes 102 can operate on the data stored in the data tables104 a in the main memory to generate data analysis results. As the OLTPand OLAP server processes 102 perform their respective operations, thein-memory computing system 100 can determine runtime information abouteach of the server processes 102 and/or the operating system of thein-memory computing system 100. In some embodiments, the runtimeinformation can include state information regarding the OLTP and/or OLAPoperations. At box 1130, the in-memory computer system 100 can store theruntime information in the main memory 104. For example, the runtimeinformation can be stored in one or more of the system tables 104 b.

In box 1140 the in-memory computer system can invoke one or more scriptprocedures stored in the main memory 104. In one embodiment, the scriptprocedures, such as SQL scripts 820, can be stored in one or more tables(e.g., system tables 104 b or scripts tables 104 e) in the main memory104. The in-memory computer system 100 can then invoke one or more ofthe script procedures to operate on the runtime information stored inthe main memory 104. In one embodiment, one script procedure can beinvoked to select runtime information from one of the system tables 104b and copy it to a statistics table 104 c. For example, a particular SQLscript can be invoked to select and gather runtime information for aparticular server process 102 stored in a corresponding system table 104b and copy it as monitoring data to one or more statistics tables 104 c.In other embodiments, the same or a different SQL script 820 can beinvoked to analyze the selected runtime information. The analysis of theruntime information can include generating system performance metricsand storing it in one of the statistics tables 104 c. In someembodiments, the SQL script may also compare the monitoring data or theperformance metrics against one or more alert condition definitions. Insuch embodiments, the logic for analyzing the monitoring data togenerate system performance metrics or comparing it according to analert condition definition can be implemented in the SQL script 820 aslogical operators included in one or more of the component SQL commands.The results of the analysis can be then be stored in the statisticstables 104 c. In embodiments, when an alert condition is detected in themonitoring data or the system performance metrics, an alert can bestored to one or more alerts tables 104 d in the main memory. In oneembodiment, actions performed at box 1140 can be achieved without any ofthe runtime or monitoring data being taken out of the main memory 104.

The above description illustrates various embodiments of the presentdisclosure along with examples of how aspects of the present disclosuremay be implemented. The above examples and embodiments should not bedeemed to be the only embodiments, and are presented to illustrate theflexibility and advantages of the present disclosure as defined by thefollowing claims. Based on the above disclosure and the followingclaims, other arrangements, embodiments, implementations and equivalentswill be evident to those skilled in the art and may be employed withoutdeparting from the spirit and scope of the disclosure as defined by theclaims.

1. (canceled)
 2. A method in a computer system comprising: performing,on runtime data generated from one or more applications in communicationwith the computer system, online transaction processing for collectingthe runtime data and online analytical processing for analyzing theruntime data; storing results of performing the online transactionprocessing and online analytical processing on the runtime data in oneor more system tables in memory of the computer system; and executing afirst script procedure on the runtime data stored in the system tablesto generate monitoring data for the computer system, wherein the firstscript procedure is stored in the memory of the computer system alongwith the runtime data, and the first script procedure is operable toperform operations on the runtime data while the runtime data resides inthe memory of the computer system.
 3. The method of claim 2 wherein theonline transaction processing and the online analytic processing areperformed concurrently on the runtime data.
 4. The method of claim 2further comprising executing a second script procedure stored in thememory of the computer system on the runtime data in the system tablesto perform data analysis of the monitoring data to generate systemperformance metrics, and to write the system performance metrics to oneor more statistics tables stored in the memory of the computer system.5. The method of claim 2 further comprising executing a third scriptprocedure stored in the memory of the computer system on the runtimedata in the system tables to evaluate the monitoring data according toone or more alert condition definitions, and to write an alert to one ormore alert tables stored in the memory of the computer system when analert condition is detected.
 6. The method of claim 2 wherein theruntime data is copied from the system tables to one or more statisticstables or one or more alert tables in the memory of the computer systemwithout inter-process communications.
 7. The method of claim 2 wherein ascheduler process is stored in the memory of the computer system alongwith one or more script procedures that operate on the runtime data inthe system tables of the memory.
 8. The method of claim 7 wherein thescheduler process executes the script procedures on a periodic orscheduled basis.
 9. The method of claim 2 wherein none of the runtimedata is removed from the memory of the computer system during datacollection or analysis processes.
 10. The method of claim 2 whereinruntime data collection processes are capable of being changed withoutaffecting alert determination processes, and the alert determinationprocesses are capable of being changed without affecting the datacollection processes.
 11. A computer system comprising: at least oneserver computer having a memory for storing one or more computerprograms for execution by the server computer, the computer programsconfigured to cause the server computer to perform operationscomprising: conducting, on runtime data generated from one or moreapplications in communication with the computer system, onlinetransaction processing for collecting the runtime data and onlineanalytical processing for analyzing the runtime data; storing results ofperforming the online transaction processing and online analyticalprocessing on the runtime data in one or more system tables in thememory of the computer system; and executing a first script procedure onthe runtime data stored in the system tables to generate monitoring datafor the computer system, wherein the first script procedure is stored inthe memory of the computer system along with the runtime data, and thefirst script procedure is operable to perform operations on the runtimedata while the runtime data resides in the memory of the computersystem.
 12. The computer system of claim 11 wherein the onlinetransaction processing and the online analytic processing are performedconcurrently on the runtime data.
 13. The computer system of claim 11wherein the operations further comprise executing a second scriptprocedure stored in the memory of the computer system on the runtimedata in the system tables to perform data analysis of the monitoringdata to generate system performance metrics, and to write the systemperformance metrics to one or more statistics tables stored in thememory of the computer system.
 14. The computer system of claim 11wherein the operations further comprise executing a third scriptprocedure stored in the memory of the computer system on the runtimedata in the system tables to evaluate the monitoring data according toone or more alert condition definitions, and to write an alert to one ormore alert tables stored in the memory of the computer system when analert condition is detected.
 15. The computer system of claim 11 whereinthe runtime data is copied from the system tables to one or morestatistics tables or one or more alert tables in the memory of thecomputer system without inter-process communications.
 16. The computersystem of claim 11 further comprising a scheduler process stored in thememory of the computer system along with one or more script proceduresthat operate on the runtime data in the system tables of the memory,wherein the scheduler process executes the script procedures on aperiodic or scheduled basis.
 17. The computer system of claim 11 whereinnone of the runtime data is removed from the memory of the computersystem during data collection or analysis processes.
 18. Anon-transitory computer readable storage medium having computer programsstored thereon, which when executed by a computer system, cause thecomputer system to perform operations comprising: conducting, on runtimedata generated from one or more applications in communication with thecomputer system, online transaction processing for collecting theruntime data and online analytical processing for analyzing the runtimedata; storing results of performing the online transaction processingand online analytical processing on the runtime data in one or moresystem tables in memory of the computer system; and executing a firstscript procedure on the runtime data stored in the system tables togenerate monitoring data for the computer system, wherein the firstscript procedure is stored in the memory of the computer system, and isoperable to perform operations on the runtime data while the runtimedata resides in the memory of the computer system.
 19. The computerreadable storage medium of claim 18 wherein the online transactionprocessing and the online analytic processing are performed concurrentlyon the runtime data.
 20. The computer readable storage medium of claim18 wherein the operations further comprise executing a second scriptprocedure stored in the memory of the computer system on the runtimedata in the system tables to perform data analysis of the monitoringdata to generate system performance metrics, and to write the systemperformance metrics to one or more statistics tables stored in thememory of the computer system.
 21. The computer readable storage mediumof claim 18 wherein the operations further comprise executing a thirdscript procedure stored in the memory of the computer system on theruntime data in the system tables to evaluate the monitoring dataaccording to one or more alert condition definitions, and to write analert to one or more alert tables stored in the memory of the computersystem when an alert condition is detected.